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The MAILING DATE of this communication appears on the cover sheet with the correspondence address » 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent temi adjustment. See 37 CFR 1.704(b). 

Status 

1 )n Responsive to communication(s) filed on . 

2a)^ This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-10 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) K Claim(s) 1-10 is/are rejected. 

7) K Claim(s) 1-10 is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

1 1) n The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) n The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-{d) or (f). 

a)nAII b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received In this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) D The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Infomrial Patent Application (PTO-152) 

3) □ Infomnation Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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Response to request for reconsideration of the application in view of the remarks 
Applicant's remarks filed July 1 1, 2003 have been fully considered but they are not 
persuasive. 

> Response to remarks on pages 1 and 2, lines 6-7 and 1-6: Applicant argues that the 

references viewed individually or in combination, do not teach or suggest applicant's 
advantageous method for capturing and processing image information. Examiner's 
reply: Ercan et al. (Ercan), Ercan's invention relates to a method of processing digital 
image information. It relates particularly but not exclusively to a method of selectively 
modifying pixel intensity information in order to improve legibility or visibiUty of parts 
of a digital image. Ercan does not explicitly specify recovering missing information, 
however, in Figs. 6A, 9A and lOA illustrate the missing information. Heller et al. 
(Heller), Heller in Fig. 6 illustrate in step 84, compensation pixel values are generated to 
replace the missing data from the defective pixels in sensor array 12. In one preferred 
embodiment, the compensation pixel values are generated by examining the pixel values 
of the pixels surrounding the defective pixels. Thus, the pixel values of each 
surrounding pixels of a defective pixel are averaged to arrive at a compensation pixel 
value. In another embodiment, the pixel value of a pixel that precedes the defective 
pixel is used as the compensation pixel value. Therefore Ercan and Heller suggest 
applicant's invention. 

> Applicant argues on page 2, lines 7-21 : that claimed method is directed to recovering 

missing color data in a two-dimensional color array. Examiner's reply: Ercan in Fig. 5, 
illustrate two-dimensional color array. 
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> Applicant argues on page 2, lines 7-14: that the first one-dimensional color recovery 

process generates intermediate second color image data from first color image data and 
the second one-dimensional color recovery process generates the desired third color 
image data fi"om second color image data. Examiner's reply: Ercan in Fig. 5 illustrate a 
mapping arrangement. For example: a single objective lens is used to provide a common 
optical path for two CCDs. The image is split using a 50% reflective mirror with 50% 
of the light going to a first CCD and 50% going to a second CCD. The comparison of 
two images is done until the desired color image data achieved. 

> Applicant argues on page 4, lines 6-14: the reference Heller et al. deal with correcting 

defective pixels on image sensing elements. Examiner's reply: interpretation: correcting 
defective pixels on image sensing elements would have the same concept as correcting 
missing information pixels on image elements. Otherwise the applicant must expUcitly 
specify the differences. 

> Applicant argues on page 4, lines 15-23: Heller et al. do not expHcitly specify using non- 

linear filters. Examiner's reply: but Ercan teach in abstract, using of non-linear 
algorithm (can be considered as non-linear filter). 

> Applicant argues on page 6, lines 1-4: Katayama is not directed to recovering missing 

color information. Examiner's reply: Katayama on page 3 paragraph 0082, in a second 
embodiment teach correcting missing color on the image, and also see Fig. 31. 

> Applicant argues on page 6, lines 8-28; regarding claims 3 and 8 under 1 12 1^^ rejection, 

which non-linear filter are rank-order filter. Examiner's reply: Applicant fails to provide 
the specification of rank-order filter (non-linear filter). The following steps will be 
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useful to eliminate the 1 12 1^* rejection, if Applicant provides the requested information: 
1- what are the assumptions for designing the non-linear filter? 2- what are the 
formulations for the non-linear filter? 3- what are the limitations for the non-linear 
filter? 

> Applicant argues on page 6, lines 8-28; regarding claims 4, 5, 9 and 10 under 112 1'^ 
rejection, that is beyond dispute that the wavelengths present in the red, green and blue 
regions of the spectrum are well known to those skilled in the art. Examiner's reply: It is 
known that fi-om 400nm-700nm consider as a visible region. The major spectral colors 
as a linear sequence fi-om red (at 700mm), orange (at - 650mm), yellow (at - 550mm), 
green (at - 520nmi), blue (at ~ 450mm), tand violet at 400 nm. Question: How does 
Applicant consider a combination of red and green wavelengths, while there are other 
colors such as orange and yellow colors between the red and green wavelengths? 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1, 3, 6 and 8 rejected under 35 U.S.C. 103(a) as being unpatentable over Ercan et al. and 
fiirther in view of Heller et al. 



1. Claim 1. 
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"A method for electronically capturing and processing image information comprising the steps of 
(a) providing a two-dimensional array of discrete image sensing elements, each discrete element 
capable of providing an electronic information signal in response to incident illumination, said 
electronic information signal corresponding to the intensity of said incident illumination, each 
discrete element being specifically responsive to one of at least three predetermined colors; (b) 
obtaining first color image data by exposing the two-dimensional array to image-information 
bearing illumination such that each discrete element provides said electronic information signal, 
said first color image data comprising the collection of said electronic information signals; (c) 
recovering missing color information along a first dimension by (i) interpolating the first color 
image data along said first dimension to provide first interpolated color data, (ii) forming a first 
difference channel between said first color image data and said first-interpolated color data, (iii) 
applying a first one-dimensional non-linear filter to said first difference channel, whereby the 
first-recovered image data is obtained as a combination of the first color image data and the 
filtered first difference channel, and iv) forming second color data comprising the first color data 
and the first recovered color data; and (d) recovering missing color information along a second 
dimension by (i) interpolating the second color image data along said second dimension to 
provide second interpolated color data, (ii) forming a second difference channel between said 
second color image data and said second interpolated color data, (iii) applying a second 
one-dimensional non-linear filter to said second difference channel, whereby the 
second-recovered color data is obtained as a combination of the second color data and the 
filtered second difference channel, and iv) forming final recovered image data comprising the 
second color data and the second recovered color data", the steps of (a) and (b) are obvious 
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because Ercan et al. invention is digital image processing. But Ercan does not explicitly specify 
the recovering missing information, however Heller et al. teaches in (col. 9, lines 1-4), in step 84, 
compensation pixel values are generated to replace the missing data from the defective pixels in 
sensor array 12. In one preferred embodiment, the compensation pixel values are generated by 
examining the pixel values of the pixels surrounding the defective pixels. Thus, the pixel values 
of each surrounding pixels of a defective pixel are averaged to arrive at a compensation pixel 
value. In another embodiment, the pixel value of a pixel that precedes the defective pixel is used 
as the compensation pixel value. 

Thus, it would have been obvious to one of ordinary skill in the art at the time the invention was 
made to incorporate the teaching of Heller into Ercan in order for interfacing with the 
programmable non-volatile memory and accessing the defective pixel location information; and 
a input/output port for communicating with control and post-processing circuitry. 

2. Claim 3. 

"The method of claim 2, wherein said first and second one-dimensional non-linear filters are 
rank-order filters", Ercan teaches in abstract a related method of processing digital image 
information involves modifying pixel intensities according to a non-linear algorithm, with the 
result that one or more regions of the image have their median pixel intensities adjusted to a level 
which is closer to the median pixel intensity for other parts of the image. 

3. Claim 6. 

"An electronic imaging apparatus comprising: a two-dimensional array of discrete image sensing 
elements for generating first color image data, each discrete element capable of providing an 
electronic information signal in response to incident illumination, said electronic information 
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signal corresponding to the intensity of said incident illumination, each discrete element being 
specifically responsive to one of at least three predetermined colors; a first color recovery 
module for generating a second color image data fi-om said first color image data, the first color 
recovery module having first means for interpolating said first color data along a first dimension 
to provide first-interpolated color data, first means for non-linear filtering and combining said 
first-interpolated color data with said first color image data in said first dimension to provide first 
recovered color data, and forming second color data comprising said first color data and said 
first-recovered data; and a second color recovery for generating a final color-recovered image 
data fi*om said second color image data, the second color recovery module having second means 
for interpolating said second color data along a second dimension to provide second interpolated 
color data, second means for non-linear filtering and combining said second, interpolated color 
data with said second color image data in said second dimension to provide a second-recovered 
color data, and forming a final recovered image, comprising said second color data and said 
second-recovered data", the steps of a two-dimensional array of discrete image and a first color 
recovery are obvious because Ercan et al. invention is digital image processing. The step of 
recovering missing color information is obvious because recovering missing color information 
should be taking along both dimensions, Ercan et al. Illustrates in Fig. 1, a typical hardware 
layout for apparatus, which uses the inventive method. A light image is detected by a charge- 
coupled device (CCD) 1, which is an element of a type, which is used in video cameras. The 
detected image is split into red, green and blue (RGB) analog streams, which are passed to 
analog to digital converters (A/D converters) 2. The digital signals so produced are passed to a 
first digital signal processor (DSP) stage 3, which in the example illustrated filters the 16-bit 
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RGB digital signals into 8-bit RGB signals using user-selectable criteria. The signals are then 
passed to a second DSP stage 4, in which RGB signals are converted to a more compact YUV 
format. The YUV signals are then passed to a third DSP stage 5, in which a data compression 
algorithm is applied. It will be appreciated that this configuration is illustrative only, and many 
different configurations can be applied to achieve the objects of the invention. 

4. Claim 8. 

"The electronic imaging apparatus of claim 7, wherein said first and second means for non-hnear 
filtering both include rank-order filters", Ercan teaches in abstract a related method of processing 
digital image information involves modifying pixel intensities according to a non-linear 
algorithm, with the result that one or more regions of the image have their median pixel 
intensities adjusted to a level which is closer to the median pixel intensity for other parts of the 
image. 

5. Claims 2 and 7 rejected under 35 U.S.C. 103(a) as being unpatentable over Ercan, and 
further in view of Katayama. 

6. Claim 2. 

"The method of claim 1, wherein the discrete elements are pattern-wise arranged such that (a) no 
two discrete elements that are contiguous along said first or second dimension are specifically 
responsive to the same one of said at least three predetermined colors, and (b) no more than one 
discrete element is contiguously between two discrete elements that are specifically responsive to 
the same one of said at least three predetermined colors", Ercan does not specify pattern-wise 
arranged, however, Katayama et al. teaches in paragraph 0021 to provide an image sensing 
apparatus which determines whether the optimum image sensing conditions are achieved or not 
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by judging whether there is a predetermined pattern in an image sensing field. Thus, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made to 
incorporate the teaching of Katayama into Ercan in order to used a background out of a plurality 
of pads with markers, thus it is possible to set the background which is most suitable to the shape 
and size, of an object whose shape is to be recognized. As a result, the image sensing 
parameters, which are most suitable to the object, can be determined. Therefore, the precision of 
the three-dimensional shape recognition on the basis of images obtained by using the optimum 
image sensing parameters can be improved. 
7. Claim 7. 

"The electronic imaging apparatus of claim 6, wherein the discrete elements are pattern- wise 
arranged such that (a) no two discrete elements that are contiguous along said first or second 
dimension are specifically responsive to the same one of said at least three predetermined colors, 
and (b) no more than one discrete element is contiguously between two discrete elements that are 
specifically responsive to the same one of said at least three predetermined colors", Ercan does 
not specify pattem-wise arranged, however, Katayama et al. teaches in paragraph 0021 to 
provide an image sensing apparatus which determines whether the optimum image sensing 
conditions are achieved or not by judging whether there is a predetermined pattern in an image 
sensing field. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Katayama into Ercan in order to used a 
background out of a plurality of pads with markers, thus it is possible to set the background 
which is most suitable to the shape and size, of an object whose shape is to be recognized. As a 
result, the image sensing parameters, which are most suitable to the object, can be determined. 
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Therefore, the precision of the three-dimensional shape recognition on the basis of images 
obtained by using the optimum image sensing parameters can be improved. 



8. Claims 3-5 and 8-10 rejected under 35 U.S.C. 1 12, first paragraph, as containing subject 
matter which was not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor(s), at the time the application was filed, had 
possession of the claimed invention. In claims 3 and 8, the appHcant should specify the 
specifications and parameters for rank-order filters (non-linear filter), and also in claims 4-5 and 
9-10 applicant should have the complete specifications of red, green and blue wavelengths. 



THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi-om the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated fi-om the mailing date of the advisory action. In no event, 



Claim Rejections - 35 USC § 112 



Conclusion 
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however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Javid A Amini whose telephone number is 703-605-4248. The 
examiner can normally be reached on 8-4pm. 

If attempts to reach the examiner by telephone are unsuccessfiil, the examiner's 
supervisor, Michael Razavi can be reached on 703-305-4713. The fax phone numbers for the 
organization where this application or proceeding is assigned are 703-746-8705 for regular 
communications and 703-746-8705 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-306-0377. 



Javid A Amini 
Examiner 
Art Unit 2672 



Javid Amini 
September 16, 2003 
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